A search engine is a program designed to find information stored on one or more computing devices. In its most recognized format, search engines, such as those employed by MSN, Yahoo, and Google, for example, are used to find websites and documents throughout the Internet. These types of search engines typically perform “horizontal” searches, providing a breadth of information related to the search query. However, users often desire information within a narrow topic or field. Accordingly, vertical search engines have been designed to perform specialized searches to mine data for a narrow niche. For example, a shopping vertical search engine may be employed to search information for products users may wish to purchase. Other types of vertical search engines include job/career verticals, travel verticals, local services verticals, research verticals, real estates verticals, automobile verticals, and news verticals, for example.
Typically, vertical search engines maintain a database containing information relating to the particular topic of the vertical. In addition, the information is typically fed to the search engine by a number of feed sources. When the vertical search engine receives search queries, it may access the database to generate search results for the query. For example, a shopping vertical search engine may include a database storing product information and offers that are fed to the search engine by a number of different sources, such as merchants and retailers. When the shopping vertical search engine receives search queries, it accesses the database and provides a listing of products with associated product information and offers based on the query.
A drawback to having only feed data available to vertical search engines is that search results provide only limited information to users. A large amount of information relating to the particular topic of the vertical may be missing. To capture more information, web crawl information may also be added to the database. For example, a shopping vertical search engine may crawl retailer websites to capture information regarding products being sold by the retailer. However, information gathered by such a web crawl may be less complete and less reliable than information fed to the search engine. In addition, it is difficult to commingle and order items from both feed sources and crawl sources such that items users are most likely to be interested in are provided first. For example, if items were ordered based on the reliability and completeness of information associated with each item, items from feed sources would typically be listed first and crawl sources would been hidden at the end of the search results, even though users may be interested in items from crawl sources.